A New Paradigm for Multimedia Information Access Through the Integration of Content, Context and Common Knowledge

We present a new paradigm for multimedia information access, named the “Holistic” information access paradigm, and its underlying architecture, the 3C (Content, context, and common-background) knowledge integration architecture. The holistic information access approach is context-sensitive, concept-based, and proactive. “Context-sensitive” means that the approach incorporates the characteristics of the information seeker (or the audience) as well as his/her intended use for the sought information. Common (domain) knowledge documents are used for characterizing the seeker as well as the information task and represented using user-independent, role-based profiles. Concept-based means that the user interacts with the system through an ontology of concepts instead of a vocabulary of words/phrases. “Proactive” means that the approach incorporates retrieval and classification in advance, based on background domain knowledge documents designated by the information seeker. Content knowledge is obtained via audio-visual analyzers such as object detectors and feature extraction modules. Context knowledge is obtained via shallow and deep text mining methods. These three components are integrated via the XML-based standard knowledge representation scheme XML Topic Maps (XTM). We demonstrate the use of the approach in an automatic presentation generation system in the context of K-12 education.